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DETAILED ACTION 



Drawings 

1 . The drawings are objected to because reference character "204" in FIG. 9 should 
be - 203 -. 

2. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they do not include the following reference sign(s) mentioned in the 
description: 

• Page 1 1 Paragraph [042] line 2: "step 310" 

• Page 14 Paragraph [052] line 2: "S 3 1 (513)" 

• Page 23 Paragraph [083] line 3: "a(1005)" 

• Page 24 Paragraph [085] lines 2-3: "A new ROI record (1007)" 

• Page 24 Paragraph [087] line 3: "last element (1 008)" 

3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: 1007 in FIG. 10F. 

4. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(4) 
because reference character "105" in FIG. 1 and FIG. 3 has been used to designate 
both "Local interface" and "Next i, j". 

5. Corrected drawing sheets in compliance with 37 CFR 1 .121 (d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
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prior version of the sheet, even if only one figure is being amended. Each drawing sheet 
submitted after the filing date of an application must be labeled in the top margin as 
either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1 .121(d). If the 
changes are not accepted by the examiner, the applicant will be notified and informed of 
any required corrective action in the next Office action. The objection to the drawings 
will not be held in abeyance. 

Specification 

6. The disclosure is objected to because of the following informalities: 

• Page 3 Paragraph [015]: "FIGs 10A-C" should be -- FIGs 10A-G -- 

• Page 5 Paragraph [021] lines 1-2: "the method for recognizing road signs 
in a digital image 108" should be - the method the method for recognizing 
road signs in a digital image 106 -- 

• Page 5 Paragraph [022] line 1 : "the method for recognizing road signs in a 
digital image 109" should be the method the method for recognizing 
road signs in a digital image 106 

• Page 6 Paragraph [025] line 4: "the method for recognizing road signs in a 
digital image 110" should be - the method the method for recognizing 
road signs in a digital image 106 - 

• Page 6 Paragraph [026] line 7: "the method for recognizing road signs in a 
digital image 111" should be - the method the method for recognizing 
road signs in a digital image 106 - 
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• Page 6 Paragraph [026] lines 2-3: "the method for recognizing road signs 
in a digital image 1 12" should be -- the method the method for recognizing 
road signs in a digital image 106 - 

• Page 7 Paragraph [027] lines 1-2: "the method for recognizing road signs 
in a digital image 113" should be - the method the method for recognizing 
road signs in a digital image 106 - 

• Page 7 Paragraph [027] lines 2-3: "the method for recognizing road signs 
in a digital image 1 14" should be - the method the method for recognizing 
road signs in a digital image 106 - 

• Page 1 1 Paragraph [042] line 4: "then processing stops at step 309" 
should be -- then processing stops at step 210 - 

• Page 14 Paragraph [050] lines 3-4: "Si = E(x 0 1 , x 0 1 ; 1, n)" should be - "Sj 
= E(x 0 \ xo 2 ; 1, n)" - 

• Page 14 Paragraph [052] line 3: "xi 1 (51 5) and right boundary X1 2 (516)" 
should be - x 0 1 (51 5) and right boundary x 0 2 (516) -- 

• Page 16 Paragraph [056] line 1 : "step 604" should be - step 605 - 

• Page 16 Paragraph [058] line 1 : "step 604" should be -- step 606 - 

• Page 16 Paragraph [058] line 2: "step 605" should be - step 608 - 

• Page 16 Paragraph [058] line 3: "step 607" should be - step 606 - 

• Page 16 Paragraph [059] line 1 : "step 608" should be - step 609 - 

• Page 16 Paragraph [059] line 3: "step 609" should be - step 610 - 



Application/Control Number: 10/696,977 Page 5 

Art Unit: 2112 

• Page 17 Paragraph [060] line 1: "step 604" first and second occurrence 
should be step 607 - 

• Page 17 Paragraph [060] line 3: "step 604" should be - step 607 - 

• Page 17 Paragraph [060] line 7: "step 610" should be - step 612 - 

• Page 19 Paragraph [068] line 4: "step 708" should be - step 709 - 

• Page 19 Paragraph [068] line 5: "step 709" should be - step 708 - 

• Page 26 Paragraph [095] line 1 : "and OCR algorithm is used" should - an 
Optical Character Recognition (OCR) is used - 

Appropriate correction is required. 

7. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01 (o). Correction 
of the following is required: 

• Claim 6 lines 1-2: "the first threshold value at a location X(i,j) is the 
average of a square submatrix centered at X(i j)." 

• Claim 1 1 lines 2 and 4: "a validating column vector", and "a validating row 
vector". 

Claim Objections 

8. Claims 8, 15, 17 and 18 are objected to because of the following informalities: 

• Claim 8 lines 4-5: "The method of claim 1 , where the matrix contains only 
binary values." should be deleted. 
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• Claim 15 line 2: "the matrix for a element" should be - the matrix for an 
element - 

• Claim 17 line 5: "logic correlate at least one region" should be - logic 
configured to correlate at least one region 

• Claim 18 line 2: "the method" should be - the system -- 

Appropriate correction is required. 

Claim Rejections - 35 USC § 101 

9. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

10. Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. The claim limitation "A computer readable 
medium having a computer program" is intended to cover electromagnetic and infrared 
propagation mediums mentioned on page 6 Paragraph [026] lines 8-9 in the 
specification. Since electromagnetic and infrared propagation mediums are not a 
tangible, physical article or object to constitute a manufacture, and it's not a machine, 
process or composition of matter, the previously mentioned claim does not fall within a 
statutory category of invention. The examiner suggests replacing the sentence "The 
computer- readable medium can be, for example but not limited to, an electronic, 
magnetic, optical, electromagnetic, infrared, or semiconductor system, system, device, 
or propagation medium." in Paragraph [026] lines 7-9 in the specification with - The 
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computer-readable medium can be, for example but not limited to, an electronic, 
magnetic, optical or semiconductor system or device. - 

Claim Rejections - 35 USC § 112 

1 1 . The following is a quotation of the second paragraph of 35 U.S.C. 1 1 2: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

12. Claims 2, 3, 5, 6, 9, 12, and 13-16 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

• Claim 2 recites the limitations "the matrix' in line 4" and "the reference 
matrix" in line 6. There is insufficient antecedent basis for this limitation in 
the claim. However, it appears to be - the at least one matrix - and - the 
template matrix - respectively, and has been treated as such. Affirmation 
of this is required by the appropriate amendment. 

• Claim 3 recites the limitations "the performing step" and "the matrix" in 
lines 1 and 2 respectively. There is insufficient antecedent basis for these 
limitations in the claim. However, it appears to be dependant upon claim 2, 
and "the matrix" appears to be - the at least one matrix --, and has been 
treated as such. Affirmation of this is required by the appropriate 
amendment. 

• Claim 5 recites the limitation "the matrix" in line 2. There is insufficient 
antecedent basis for this limitation in the claim. 
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• Claim 6 is dependant upon claim 5. 

• Claim 9 recited the limitation "the matrix" in line 1 . There is insufficient 
antecedent basis for this limitation in the claim. However, it appears to be - 

- the template matrix - and has been treated as such. Affirmation of this is 
required by the appropriate amendment. 

• Claim 10 recites the limitation "the matrix" in line 1 . There is insufficient 
antecedent basis for this limitation in the claim. 

• Claim 12 recites the limitation "the submatrix" in line 2. There is insufficient 
antecedent basis for this limitation in the claim. However, it appears to be 

- the at least one submatrix - and has been treated as such. Affirmation 
of this is required by the appropriate amendment. 

• Claim 13 recites the limitation "the submatrix" in line 3. There is insufficient 
antecedent basis for this limitation in the claim. However, it appears to be 

- the at least one submatrix - and dependant upon claim 2. It has been 
treated as such. Affirmation of this is required by the appropriate 
amendment. 

• Claim 14 recites the same limitations of claim 13 and has been considered 
a duplicate. It is redundant and should be deleted. 

• Claim 15 recites the limitations "the extracting step" and "the matrix" in 
lines 1 and 2 respectively. There is insufficient antecedent basis for these 
limitations in the claim. However, it appears to be dependant upon claim 2 
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and to be - the at least one matrix - and has been treated as such. 
Affirmation of this is required by the appropriate amendment. 
• Claim 16 recites the limitation "the extracting step" in line 1 . There is 
insufficient antecedent basis for this limitation in the claim. 
Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

13. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

14. Claims 1,2, 7, 13, 14, 17 and 18 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Berube et al. (U.S. 7,130,454 B1). 

Re claim 1 : Berube et al. disclose a method for recognizing a road sign in a 
digital color image, where the road sign is associated with a shape template and 
at least one color criterion (Col. 2 lines 44-47 and Col. 7 lines 33-34), the method 
comprising: capturing a digital color image (image acquisition stage 22, Col. 7 
lines 45-49); correlating at least one region of interest within the digital color 
image with a template matrix (Col 16 lines 3-7), where the template matrix 
(Reference Set) is specific to a reference sign (Col. 15 lines 61-67); and 
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recognizing the image as containing the reference sign, responsive to the 
correlating step (Col.15 lines 21-26). 

Re claim 2, as understood: Berube et al. disclose performing color segmentation 
on the digital color image to produce at least one matrix (Col. 9 lines 30-33 and 
44-46); extracting at least one region of interest from the matrix (area of pixels) to 
produce at least one submatrix (blob) containing at least one potential road sign 
(Col. 9 lines 31-33); correlating the at least one submatrix with a template matrix 
to produce a correlation coefficient, where the reference matrix is specific to a 
reference sign (Col. 16 lines 7-18); and recognizing the image as containing the 
reference sign, based upon a comparison of the correlation coefficient and a 
correlation threshold value (Col. 16 lines 20-25). 

Re claim 7: Berube et al. disclose a second color criterion (different image 
reference sets), where the first criterion is used in dim light conditions and the 
second criterion is used in bright light conditions (Col. 15 lines 4-11). 

Re claims 13 and 14: Berube et al. disclose calculating a two-dimensional 
correlation coefficient which measures the correlation between the submatrix and 
a two-dimensional matrix (Col. 16 lines 7-18). 
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Re claim 17: Berube et al. disclose a computer readable medium having a 
computer program for recognizing a road sign in a digital color image, where the 
road sign is associated with a shape template and at least one color criterion 
(Col. 2 lines 44-47 and Col. 7 lines 33-36), comprising: logic configured to 
capture a digital color image (Col. 3 lines 10-12 and image acquisition stage 22, 
Col. 7 lines 45-49); logic correlate at least one region of interest within the digital 
color image with a template matrix (Col 16 lines 3-7), where the template matrix 
is specific to a reference sign (Col. 15 lines 61-67); and logic configured to 
recognize the image as containing the reference sign, responsive to the 
correlation logic (Col. 15 lines 21-26). 

Re claim 18: Berube et al. disclose a system for recognizing a road sign in a 
digital color image, where the toad sign is associated with a shape template and 
at least one color criterion (Col. 2 lines 44-47 and Col. 7 lines 33-34), the method 
comprising: means for capturing a digital color image (Col. 3 lines 10-12); means 
for correlating at least one region of interest within the digital color image with a 
template matrix, where the reference matrix is specific to a reference sign (Col. 
15 lines 28-29 and 47-53); and means for recognizing the image as containing 
the reference sign, responsive to the correlation means (Col. 15 lines 21-26). 
Claim 17 invokes 35 U.S.C. 112 sixth paragraph. 
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Claim Rejections - 35 USC § 103 

15. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

16. Claim 3, as understood is rejected under 35 U.S.C. 103(a) as being unpatentable 
over Berube et al. in view of Laumeyer et al. (U.S. 6,453,056 B2). The teachings of 
Berube et al. have been discussed above. 

However, Berube et al. fails to disclose setting each element of the matrix to a 
first value if the corresponding pixel position in the digital color image matches any of 
the at least one color criterion associated with the road sign. 

Laumeyer et al. teaches setting each element of the matrix to a first value if the 
corresponding pixel position in the digital color image matches any of the at least one 
color criterion associated with the road sign (Col. 1 1 lines 27-30 and Col. 12 lines 26- 
30). 

Therefore, in view of Laumeyer et al., it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Berube et al.'s 
system by including the method step of setting each element of the matrix to a first 
value if the corresponding pixel position in the digital color image matches any of the at 
least one color criterion associated with the road sign in order to create a graphic-based 
search engine in addition of the road sign detection. 
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17. Claims 4 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Berube et al. in view of Lin et al. (U.S. 7,039,229 B2). The teachings of Berube et al. 
have been discussed above. 

However, Berube et al. fails to disclose a first/second color selected from R, G, B 
being greater than a first/thrid threshold value; and a saturation value for the 
first/second color being greater than a second/fouth threshold value where the 
saturation value is calculated using a hue, saturation and intensity model. 

Lin et al. teaches a first/second color selected from R, G, B being greater than a 
first/third threshold value (Col. 4 lines 57-61); and a saturation value for the first/second 
color being greater than a second/fourth threshold value is calculated using a hue, 
saturation and intensity model (Col. 24 lines 5-7) as recited in claims 4 and 8 
respectively . 

Therefore, in view of Lin et al., it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Berube et al.'s system by 
including a first and second color selected from R, G, B being greater than a first and 
third threshold value and a saturation value for the first and second color being greater 
than a second and fourth threshold value is calculated using a hue, saturation and 
intensity model in order to more effectively characterize and compare the colors in the 
image. 

18. Claims 5 and 6 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Berube et el. as modified by Lin et al. as applied to claim 4 above, and further in view of 
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Nakajima et al. (U.S. 6,285,778 B1). The teachings of Berube et al. modified by Lin et 
al. have been discussed above. 

However, Berube et al. modified by Lin et al. fails to disclose the at least one 
color criterion is locally adaptive such that the first threshold value varies at each 
location within the matrix and the first threshold value at a location X(i,j) is the average 
of a square submatrix centered at X(ij). 

Nakajima et al. teaches the at least one color criterion is locally adaptive such 
that the first threshold value varies' at each location within the matrix and the first 
threshold value at a location X(i,j) is the average of a square submatrix (when m = n) 
centered at X(i,j) (Col. 5 lines 13-20) as recited in claims 5 and 6 respectively . 

Therefore, in view of Nakajima et al., it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to further modify Berube et 
al.'s system as modified by Lin et al. by including the at least one color criterion is 
locally adaptive such that the first threshold value varies at each location within the 
matrix and the first threshold value at a location X(i,j) is the average of a square 
submatrix centered at X(i,j) in order to more efficiently detect and measure obstacles 
size and position. 

19. Claim 9, as understood, is rejected under 35 U.S.C. 103(a) as being 
unpatentable over Berube et al. in view of Michael et al. (U.S. 6,539,107 B1). The 
teachings of Berube et al. have been discussed above. 

However, Berube et al. fails to disclose the matrix contains only binary values. 
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Michael et al. teaches the matrix (Fig. 7) contains only binary values (Col. 7 lines 
54-56). 

Therefore, in view of Michael et al., it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Berube et al.'s system by 
including the matrix which contains only binary values in order to save processing time. 

20. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Berube 
et al. in view of Kim et al. (U.S. 2001/0040982 A1 ). The teachings of Berube et al. have 
been discussed above. 

However, Berube et al. fails to disclose recursively removing any invalid row and 
any invalid column from the matrix, where an invalid row contains less than a first 
threshold value of a binary value and an invalid column contains less than a second 
threshold value of the binary value. 

Kim et al. teaches recursively removing any invalid row and any invalid column 
from the matrix (RECT[3]), where an invalid row contains less than a first threshold 
value of a binary value and an invalid column contains less than a second threshold 
value of the binary value (Paragraph [0050]). 

Therefore, in view of Kim et al., it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Berube et al.'s system by 
including recursively removing any invalid row and any invalid column from the matrix, 
where an invalid row contains less than a first threshold value of a binary value and an 
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invalid column contains less than a second threshold value of the binary value in order 
to optimize the minimum area to become the shape of the object. 

21 . Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Berube 
et al. in view of Gersten (U.S. 6,928,193 B2). The teachings of Berube et al. have been 
discussed above. 

However, Berube et al. fails to disclose creating a validating column vector 
corresponding to all columns in the submatrix; creating a validating row vector 
corresponding to all rows in the submatrix; correlating the validating column vector with 
a template column vector; and correlating the validating row vector with a template row 
vector. 

Gersten teaches creating a validating column vector (result vector) 
corresponding to all columns in the submatrix (image); creating a validating row vector 
corresponding to all rows in the submatrix; correlating the validating column vector (next 
FFT vector) with a template column vector; and correlating the validating row vector with 
a template row vector (Col. 3 lines 21-26, 35-37 and 45-47). 

Therefore, in view of Gersten, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Berube et al.'s system by 
including creating a validating column vector corresponding to all columns in the 
submatrix; creating a validating row vector corresponding to all rows in the submatrix; 
correlating the validating column vector with a template column vector; and correlating 
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the validating row vector with a template row vector in order to produce a corrected 
image for further processing. 

22. Claim 12, as understood, is rejected under 35 U.S.C. 103(a) as being 
unpatentable over Berube et al. in view of Oosawa (U.S. 2001/0048757 A1 ). The 
teachings of Berube et al. have been discussed above. 

However, Berube et al. fails to disclose normalizing the submatrix, such that each 
element in the normalized submatrix is the weighted sum of the element's four 
neighboring elements. 

Oosawa teaches normalizing the submatrix (P2), such that each element in the 
normalized submatrix is the weighted sum of the element's four neighboring elements 
(Paragraph [0124] lines 1-8 and 31-32, and Paragraph [0125] lines 1-6). 

Therefore, in view of Oosawa, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Berube et al.'s system by 
including normalizing the submatrix, such that each element in the normalized 
submatrix is the weighted sum of the element's four neighboring elements in order to 
perform local position matching processing within the original image. 

23. Claims 15 and 16, as understood are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Berube et al. in view of Xia et al. (U.S. 2001/0036300 A1). The 
teachings of Berube et al. have been discussed above. 
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However, Berube et al. fails to disclose scanning the matrix for a element with 
value 1 at position E(x,y); and performing a depth-first-search to find all elements with 
value 1 that are connected to E(x,y); and setting the element at position E(x,y) to a 
unique identifier; and setting the value of the connected elements to the unique 
identifier. 

Xia et al. teaches scanning the matrix for a element with value 1 at position 
E(x,y); and performing a depth-first-search to find all elements with value 1 that are 
connected to E(x,y); and setting the element at position E(x,y) to a unique identifier 
("0"); and setting the value of the connected elements to the unique identifier (Fig. 2 and 
Paragraphs [0055]-[0069]) as recited in claims 15 and 16 respectively . 

Therefore, in view of Xia et al., it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Berube et al.'s system by 
including scanning the matrix for a element with value 1 at position E(x,y); and 
performing a depth-first-search to find all elements with value 1 that are connected to 
E(x,y); and setting the element at position E(x,y) to a unique identifier; and setting the 
value of the connected elements to the unique identifier in order to reproduce a better 
image that produces a better response with respect a predetermined threshold. 

Conclusion 

24. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Javidi et al. disclose a method and system for image processing 
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for automatic road sign recognition, and Janssen et al. disclose a device and method for 
recognizing traffic signs. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jose M. Torres whose telephone number is 571-270- 
1356. The examiner can normally be reached on Monday thru Friday: 8:00am - 
4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jong-Suk (James) Lee can be reached on 571-272-7044. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

JMT 

11/08/2006 
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